Self-Instruct: Aligning Language Models with Self-Generated Instructions

We introduce Self-Instruct, a framework for improving the instruction-following capabilities of pretrained language models by bootstrapping off their own generations.

https://github.com/yizhongw/self-instruct/blob/main/docs/pipeline.JPG?raw=true

Figure 2

Figure 1とTable 10にGPT3で生成した例

GPT-3を使い、175個のシードタスクから52kの指示、82kの応答からなるデータを自動構築

Figure 6

GPT-3 Self-InstructがInstructGPTに近づいた